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C^ I Abstract 

-)— > i 

We give meaning to the first and second laws of tliermodynamics in case of 
C^ . mesoscopic out-of-equilibriuni systems which are driven by diffusion-type, specif- 

H I ically Smoluchowski, processes. The notion of entropy production is analyzed. 

The role of the Helmholtz extremum principle is contrasted to that of the more 
^ . familiar entropy extremum principles. 
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00 ■ 1 Introduction 

m 

'nI" ■ We aim at a consistent thermodynamic description of diffusion-type processes which 

'\0 I model the dynamics of non-equihbrium systems at the mesoscopic scale, [31 El HI [5l [I] . 



It is known that given the equilibrium properties of a mesoscopic (molecular) system, 
it is possible to deduce a stochastic nonequilibrium, albeit near-equihbrium, dynamics 
in terms of Fokker-Planck equations and their probability density solutions, [1] . 
Ch ! We basically go in reverse and abandon any prescribed concept of local or global 

Q ! equilibrium and ask for these thermodynamic properties that give account of a conver- 

gence (if any, this porperty is not automatically granted) towards an equilibrium state, 
/\ • even if initially a system is arbitrarily far from equilibrium, [HI |T2l |13]. Our focus is 

c^ I on a quantitative description of energy (heat, work, entropy and entropy production) 

transfer time rates in the mean, between a particle and its thermal environment. 

We explore the extremum principles which are responsible for the large time asymp- 
totic of the process, [6j. Thermodynamic function(al)s, like e.g. an internal energy, 
Helmholtz free energy and Gibbs-Shannon entropy are inferred, through suitable av- 
eraging, from the time-dependent continuous probability densities, [Sj El [TOl [1] and 
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[m [m [121 [13] • Assuming appropriate (natural) boundary data we demonstrate 
that generically the corresponding extremum principle amounts to minimizing the 
Helmholtz free energy of random motion, see also [3] . 

The following hierarchy of thermodynamical systems is adopted: isolated with no 
energy and matter exchange with the environment, closed with the energy but no 
matter exchange and open where energy-matter exchange is unrestricted. With the 
standard text-book wisdom in mind that all isolated systems evolve to the state of 
equilibrium in which the entropy reaches its maximal value, we focus our further at- 
tention on closed random systems and their somewhat different asymptotic features. 

A concise resume of a non-equilibrium thermodynamics of closed systems comprises 
the the basic conservation laws for the time rates of internal energy, heat, work and 
entropy exchange. The energy conservation implies the /^* law of thermodynamics: an 
internal energy U changes by dU in time dt, according to 

dU = 5Q + 5W (1) 

where we distinguish the imperfect differentials by 5. Normally (which will not neces- 
sarily be the case in our further discussion) one interprets dU as an increase in internal 
energy of the system due to absorbed heat 5Q > and work 6W < performed by 
the system upon its environment. 

The 11"^"^ law correlates the time rates of entropy, entropy production and heat 
exchange between the system and its environment: 

S={S)int + {S)e,t. (2) 

The entropy time rate of change is here manifestly decomposed into two contributions: 
{S)int is induced by irreversible processes that are intrinsic to the system, while {S)ext 
refers to an energy exchange between the system and its environment. 

Since {S)int > 0, this entropy production term is interpreted as the major signature 
of the //"'^ law, quite apart form its specific verbal formulation. The remaining {S)ext 
term is related to the heat exchange via {S)extdt = 6Q/T, where T is the temperature, 

PHI- 

We emphasize that neither heat nor work can be interpreted as legitimate ther- 
modynamic functions. Moreover, the very notion of entropy, sometimes viewed as a 
fundamental thermodynamic quantity, appears to be a secondary - derived notion. In 
the forthcoming statistical description, this issue will become straightforward, once we 
shall relate probabilities and statistics of random events to the (information) entropy 
notion. 

At this point there is no mention of stationary or steady states, nor any restriction 
upon the speed of involved, basically irreversible dynamical process. For the record. 



we indicate that in case of a reversible process we would have {S)int = so that an 
overall entropy change would arise solely due to the flow of heat. 

Thermo dynamical extremum principles are usually invoked in connection with the 
large time behavior of irreversible processes. One looks for direct realizations of the 
entropy growth paradigm, undoubtedly valid for isolated systems, [H], compare e. g. 
also a collection of various entropy optimization strategies in Ref . [17] . 

Among a number of admissible thermodynamic extremum principles, just for refer- 
ence in the present context, we single out a specific one. If the temperature T and the 
available volume V are kept constant, then the minimum of the Helmholtz free energy 
F = U — TS is preferred in the course of the system evolution in time, and there holds 
F = -T{S),^t < 0. 

2 Randomness vs uncertainty: Boltzmann and Gibbs- 
Shannon entropies 

We know that a result of an observation of any random phenomenon cannot be pre- 
dicted a priori (i.e. before an observation), hence it is natural to quantify an uncertainty 
of this phenomenon. Let us consider fi = (/xi, ...,fiN) as a probability measure on A^ 
distinct (discrete) events Aj,l < j < N pertaining to a model system. Assume that 
Yli=i f^j — 1 and fij = prob{Aj) stands for a probability for an event Aj to occur in 
the game of chance with A^ possible outcomes. 

The expression, whose functional (logarithmic term) provenance may be traced 
back to the thermodynamical notion of Gibbs entropy, 

N 
'5(/i) = - XI ^i ^^ ^J^ (2) 

stands for the measure of the mean uncertainty of the possible outcome of the game of 
chance and at the same time quantifies the mean information which is accessible from 
an experiment (i.e. actually playing the game). 

If we identify random event values Ai,...,Ai\f with labels for particular discrete 
"states" of the system, we may interpret Eq. ([3]) as a measure of uncertainty of the 
"state" of the system, before this particular "state" it is chosen out of the set of all 
admissible ones. This well conforms with the standard meaning attributed to the 
Shannon information entropy: it is a measure of the degree of ignorance concerning 
which possibility (event Aj) may hold true in the set {Ai,A2, ...,An} with a given a 
priori probability distribution {/ii, ...,/iAr}. 

Notice that: 

< S{fi) <\nN (4) 



ranges from certainty (one entry whose probability equals 1 and thus no information 
is missing) to maximum uncertainty when a uniform distribution fij = 1/N for all 
1 ^ J < ^ occurs. In the latter situation, all events (or measurement outcomes) are 
equiprobable and log A^ sets maximum for a measure of the "missing information". 

By looking at all intermediate levels of randomness allowed by the inequalities 
Eq. (IH]) we realize that the lower is the Shannon entropy the less information about 
"states" of the system we are missing, i.e. we have more information about the system. 
If the Shannon entropy increases, we actually loose an information available about 
the system. Consequently, the difference between two uncertainty measures can be 
interpreted as an information gain or loss. 

Anticipating various thermodynamic connotations (c.f. Boltzmann and Gibbs en- 
tropy notions) we must be careful while introducing (potentially obvious) notions of 
events, states, microstates and macrostates of a physical (or biological) system, cf . [13] . 
The celebrated Boltzmann formula 

S = kB\nW = -kB\nP (5) 

sets a link of entropy of the (thermodynamical) system with the probability P = 1/W 
that an appropriate "statistical microstate" can occur. Here, W stands for a number 
of all possible (equiprobable) microstates that imply the prescribed macroscopic (e.g. 
thermodynamical) behavior corresponding to a fixed value of S. 

It is instructive to recall that if P is a probability of an event i.e. of a particular 
microstate, then — InP (actually, with logg instead of In) may be interpreted as "a 
measure of information produced when one message is chosen from the set, all choices 
being equally likely" ("message" to be identified with a "microstate"). Another inter- 
pretation of — In P is that of a degree of uncertainty in the trial experiment. 

Remark 1: As a pedestrian illustration let us invoke a classic example of a molec- 
ular gas in a box which is divided into two halves denoted "1" and "2". We allow 
the molecules to be in one of two elementary states: Ai if a molecule can be found 
in "1" half-box and A2 if it placed in another half "2". Let us consider a particular 
n-th macrostate of a molecular gas comprising a total of G molecules in a box, with n 
molecules in the state Ai and G — n molecules in the state A2. The total number of 
ways in which G molecules can be distributed between two halves of the box in this pre- 
scribed macrostate, i.e. the number W = W{n) of distinct equiprobable microstates, 
clearly is W{n) = G\/[n\{G — n)\]. Here, P{n) = 1/W{n) is a probability with which 
any of microstates may occur in a system bound to "live" in a given macrostate. The 
maximum of W{n) and thus of kBlnW{n) corresponds to Ni = N2 = n, see e.g. at 
the "dog-flea" model discussion [T6]. 

To get a better insight into the information-uncertainty intertwine, let us consider 



an ensemble of finite systems which are allowed to appear in any of A^ > distinct 
elementary states. The meaning of "state" is left unspecified, although an "alphabet" 
letter may be invoked for convenience. 

Let us pick up randomly a large sample composed of G ^ 1 single systems, each one 
in a certain (randomly assigned) state. We record frequencies rii/G = pi, ...,nN/G = 
Pn with which the elementary states of the type 1, ..., A^ do actually occur. This sample 
is a substitute for a "message" or a "statistical microstate" in the previous discussion. 

Next, we identify the number of all possible samples of that fixed size G which 
would show up the very same statistics pi,...,pN of elementary states. We interpret 
those samples to display the same "macroscopic behavior". 

It was the major discovery due to Shannon that the number W of relevant "micro- 
scopic states" can be approximately read out from each single sample and is directly 
related to the the introduced a priori probability measure /zi, ...,/iAf, with an identifi- 
cation Pi = fii for all 1 < i < A^, by the Shannon formula: 

N 

InPy ~-G^Pilnpi = -G-5(^) (6) 

On the basis of this formula, we can consistently introduce S{fi) as the mean infor- 
mation per each (i-th) elementary state of the A^-state system, as encoded in a given 
sample whose size G ^ 1 is sufficiently large. 

By pursuing the Shannon's communication theory track, [13], we can identify states 
of the model system with "messages" (strings) of an arbitrary length G > which are 
entirely composed by means of the prescribed A^ "alphabet" entries (e.g. events or 
alphabet letters Aj with the previous probability measure /i). Then, Eq. (Q may be 
interpreted as a measure of information per alphabet letter, obtained after a particular 
message (string = state of the model system) has been received or measured, c.f. our 
discussion preceding Eq. ([6]). 

In this case, the Gibbs-Shannon entropy (by historical reasons we rename Shannon's 
iS(/i) the Gibbs-Shannon) interpolates between a maximal information (one certain 
event) and a minimal information (uniform distribution), cf. Eq. (j4]). The above 
discussion may serve as a useful introduction to an issue of the Shannon information 
workings in genomes and DNA sequences, p^j 

Till now, we have considered discrete probability distributions and their uncer- 
tainty/ delocalizat ion measures (Gibbs-Shannon entropy). The main objective of the 
present paper is a discussion of the temporal behavior of Gibbs-Shannon entropy of a 
continuous probability distribution. 

We shall focus on continuous probability distributions on R^. The corresponding 



Gibbs-Shannon entropy is introduced as follows: 

/ p{s) (is = 1 — > S{p) = — p{s) \np{s)dx (7) 

At this point it is instructive to mention that in the realistic (data analysis) frame- 
work, one encounters discrete probability data that are inferred from frequency statis- 
tics, encoded in various histograms. Definitely, there are no continuous probability 
densities at work. They typically appear as computationally useful continuous approx- 
imations of discrete probability measures. 

The situation becomes involved in case of the corresponding Gibbs-Shannon en- 
tropies, where the approximation issue is delicate. Even if one follows a pedestrian 
reasoning, we can firmly justify and keep under control the limiting behavior, fi9\ [TT] : 

N 



2_] fj'j = ^ ^ / pdx = 1 . 



An immediate question is: what can be said about the mutual relationship of S{iJ,) = 
— J2i Aijlii/Uj and S{p) = — f p{s) In p{s)ds ? 

We first observe that < — ^^ pjln pj < In A^ and consider an interval of length L 
on a line with the a priori chosen partition unit As = L/N. Next, we define: pj = pjAs 
and notice that (formally, we bypass an issue of dimensional quantities) 

S{p) = -J2{As)p,\np,-\n{As) (9) 

j 

Let us fix L and allow A^ to grow, so that As decreases and the partition becomes 
finer. Then 

In(As) < -^(As)pjlnpj < InL (10) 



where 



S{p) + \n{As) = - J](As)p^lnpj => S{p) = - f p{s) In p{s)ds (11) 



S{p) is the Shannon information entropy for the probability measure on the interval 
L. In the infinite volume L —>■ oo and infinitesimal grating As — > limits, the density 
functional S{p) may be unbounded both from below and above, even non-existent, and 
seems to have lost any computationally useful link with its coarse-grained version S{p). 

However, the situation is not that bad, if we invoke standard methods p^ [TT] 
to overcome a dimensional difficulty, inherent in the very definition of S{p), while 
admitting dimensional units. Namely, we can from the start take a (sufficiently small) 
partition unit As to have dimensions of length. We allow s to carry length dimension 



as well. Then, the dimensionless expression for the Shannon entropy of a continuous 
probability distribution reads: 



Sa{p) = - I p{s) ln[As ■ p{s)]ds (12) 



and all of a sudden, a comparison of S{p) and its coarse-grained version S{fi)) appears 
to make sense. We can legitimately set estimates for \S{fi) — iS'a(p)| and directly verify 
the approximation validity of S{fi) for a discrete probability distribution, in terms 
of the entropy Sa{p) for a As-rescaled continuous probability distribution, when the 
partition becomes finer. 

Remark 2: The value of S{pa) is a-independent if we consider Pa{x) = p{x — a). 
This reflects the translational invariance of the Shannon information measure. Let 
us furthermore investigate an effect of the scaling transformation. We denote pa,i3 = 
P p[P{x — a)], where a > 0, P > 0. The respective Shannon entropy reads: S{pa,^) = 
S{p) — In/?. An adjustment f3 = As sets an obvious link with our previous discussion. 

Remark 3: In the present paper we are interested in properties of various con- 
tinuous probability distributions, and not their coarse-grained versions. Therefore our 
further discussion will be devoid of any dimensional or partition unit connotations. 
Since negative values of the Shannon entropy are now admitted, instead of calling it 
an information measure, we prefer to tell about a "probability localization measure", 
"measure of surprise" or "measure of information deficit". 

3 Helmholtz free energy and its extremum 

Consider an equilibrium state in statistical mechanics, with (3 as an inverse temper- 
ature. As the i-th microstate we take an energy (level) Ei, i & I, with a statistical 
(Boltzmann) weight exp{—f3Ei). The macrostate is introduced as follows: choose a 
sample E = {-Ej^, Ei^, ..., Ei^, ...} and define the associated 

F{(3) = -^ln Z{(3) (13) 

with a statistical sum (partition function) Z 

Z{P) = J2^M-PE,). (14) 

E 

An internal energy reads 

U = -^ln^(/3) = (E) =Y,E^eM-m) (15) 

i 

while an entropy notion S with T = 1/(3 appears through 

U-F = TS (16) 



The " maximum entropy principle" may be replaced by (or in the least-rewritten as) 
the "principle of minimum free energy". Indeed, let Pi be a probability of occurrence 
of a microstate Ei in the macrostate configuration E, ^pj = 1. A minimum of 

F = U- 13-^S = F\p\ = J2iP^E^ + ^P^ Inp.) (17) 



P' 



is achieved for a canonical distribution: 



p^ = ^expi-(3E,). (18) 

Define S[p] = —^pilnpi and U = ^EiPi. In order to get an equilibrium distri- 
bution associated with the Shannon-Boltzman-Gibbs entropy S*, we need to extremize 
the functional: 

^p\ = -^pMPi-Ci^Pi- P^EiPi (19) 

where a and (3 are the Lagrange multipliers. We have {p* denotes an equilibrium 
probability, e.g. an ultimate solution) 

5^[p] = = [- Inp* - 1 - a - (3E,]8p, (20) 

(with arbitrary variations 5pi). Multiply the result by Pi, sum up, use the constraints 
(normalization and the fixed internal energy value) -^ 

a + l = S,-pU, (21) 

p* = exp[-S, + pU,] exp{-pE,) = exp/5(F, - E^) = ^ exp(-/3E,) . 
Notice that we deal here with a discrete probability measure, i.e. the set of p*'s such 

that J2Pi = 1- 

S** is the Shannon entropy of this discrete probability measure. In view of F = 
U — jS^^S, the Shannon entropy actually has been maximized under the normalization 
(probability measure) and fixed internal energy constraints. To be sure that the above 
F* is indeed a minimum, let us consider the relative Kullback-Leibler entropy: 

K{p,q) = J2p^H-) (22) 

and use the measure p^, = {p*} as the reference one (e g. q). We have ( K is a convex 
function with a minimum at 0): 

Kip, p.) = -S- J2pi[-S. + (3U, - (3Ei] = (3{F - F,) > (23) 

as anticipated before. 

In case of discrete probability distributions, in view of Eq. (16), a minimum of F 
is achieved in conjunction with a maximum for S. In below, we shall demostrate that 
such property is not a generic feature when continuous probability distributions come 
into consideration. 

8 



4 Thermodynamics of random phase-space motion 

Now we pass to a detailed investigation of time-dependent continuous probability dis- 
tributions and the large time behavior of their entropies. Let us begin from a concise 
resume of the (non-equilibrium) thermodynamics of closed but non-isolated systems. 
The laws of thermodynamics may be reproduced in the form|7|: dU = 6Q + SW and 
dS = dintS + dextS, where dintS > and d^xtS = 6Q/T. 

With respect to the large time behavior, the following extremum principles for ir- 
reversible processes are typically invoked: 

(1) U and V (volume) constant — *> maximum of entropy is preferred: d^tS = TdS — 
5Q > 0, together with a minimum for the entropy production: J^ ( '2l ) < 

(2) 5* and V constant — > minimum internal energy is preferred: dU = —TdintS < 0. 

(3) T and V constant — *• minimum oi F = U — TS (Helmholtz free energy) is preferred: 
dF = -TdintS < 0. 

(4) Further principles refer to the minimum of the Gibbs free energy and this of en- 
thalpy (we skip them). 

The Helmholtz extremum principle will be of utmost importance in our further discus- 
sion, as opposed to more traditional min/max entropy principles. 

We are interested not only in the existence of an extremal probability density, 
but also at an approach of p{x, t) towards such a stationary density in the course of 
time. Then the varied time dependent properties of the Helmholtz free energy, Gibbs- 
Shannon and KuUback-Leibler entropies will be of interest. 

Let us consider a phase-space diffusion process governed by the Langevin equation: 

mx + m'yx = -W{x,t)+^{t) (24) 

with standard assumptions about properties of the white noise: (^(t)) = 0, (^(i)^(i')) = 



\/2mrfkBT 6{t — t'). Accordingly, the pertinent phase-space density / = f{x,u,t) is a 
solution of the Fokker- Planck- Kramers equation with suitable initial data: 

^J{x,u,t)= (25) 



d d f 1 ^^,, A ^ksT d 



T^u + ^\iu + —W{x, t) + 



^2 



f 



dx du \ m J m du^ 

Let us define the Gibbs-Shannon entropy S = S{t) of a continuous probability distri- 
bution : 

S{t) = - f dxduflnf = -(In/) 

(By dimensional reasons we should insert a factor h with physical dimensions of the 
action under the logarithm, i.e. use ln{hf) instead of In/, but since we shall ultimately 
work with time derivatives, this step may be safely skipped.) 



An internal energy U of the stochastic process reads 



E{x,u,t) 



rau 



+ V{x,t)^U = {E) 



and the I law takes the form 



T{S),,t + {dtV) = U 



(26) 



where (dtV), if positive, is interpreted as the time rate of work externally performed 

upon the system. If negative, then we would deal with work performed by the system. 

Furthermore, let us introduce an obvious analog of the Helmholtz free energy: 



F= {E + kBTlnf) = U-TS 



so that 



F - (dtV) = T{S)e,t -TS= ~T{S),nt < . (27) 

The above result is a direct consequence of the Kramers equation. Under suitable 
assumptions concerning the proper behavior of f{x, u, t) at x, u integration boundaries 
(sufficiently rapid decay at infinities) we have jl] 

T{S)ext = liksT - {mu^)) 
ksTjdlnf 



S 
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771 



du 



) 



and thence, the 11"^'^ law 



^1 
m 



ou 



) = -T(S),„i<0. 



(2J 



As a byproduct of the discussion we have F < (dtV). For time-independent V = 
V{x) we deal with the standard F-theorem (the extremum principle pertains to the 
Helmholtz free energy F which is minimized in the course of random motion. 

The above discussion encompasses both the forced and unforced (free) Brownian 
motion. When V{x) = 0, then no asymptotic state of equilibrium (represented by 
a probability density) is accessible, the motion is sweeping. In the forced case, we 
assume a priori an existence of a unique stationary state, c.f. [HI [15], for the above 
phase-space random dynamics: 



f4x,u) = -exp 



E{x,u) 
ksT 



In this case, the time rate of the conditional Kullback-Leibler entropy: 



Hc(/t|/*) 



/ 

/In —dxdu 
J* 



(29) 
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directly appears in the F-theorem: 

kBTH, = -F = +T{S),nt > (30) 

The (negative definite) conditional entropy grows monotonically towards its maximum 
at 0. Notice that {S)int > 0, but neither {dtV) nor S need to be positive and may show 
quite complicated patterns of temporal behavior, [HI [T5] and [HI [T2]. (Both f^ and 
Tic are non-existent in case of free Brownian motion. 

Let us point out that the above discussion is sufficiently general to include a number 
of currently fashionable problems, like e.g. that of molecular motors. To see an obvious 
link it suffices to mention a typical "Brownian motor input" i.e. an explicit functional 
form of the time- dependent driving component of the exerted force and its conservative 
term in Eq. (12^ . c.f. |20j. As an example we may consider: 

mot + m-fx = -VV{x, t) + a cos{nt) + F + ^{t) (31) 

where F is a constant external force, and the spatially periodic rachet (broken reflection 
symmetry) potential V{x) is adopted. An example of the ratchet potential is: V{x) = 
Vo[sm(27rx) + ci sin(47ra;) + C2 sin(67rx)]. 

5 Thermodynamics of the Smoluchowski process 

Analogous thermodynamical features are encountered in spatial random motions, like 
e.g. standard Smoluchowski processes and their generalizations. Let us consider 

X = b{x,t) + A{t) (32) 

with {A{s)) = , {A{s)A{s')) = V2D6{s - s'). 

Given an initial probability density po{x). We know that the diffusion process drives 
this density in accordance with the Fokker-Planck equation 

dtp = DAp - V ■ (bp) . (33) 

We introduce u = Dlnp and v = b — u which obeys dtp = — V(pf). 
The Gibbs-Shannon entropy of p 

S{t) = -{\np) (34) 

typically is not a conserved quantity. We impose boundary restrictions that p, vp, bp 
vanish at spatial infinities or other integration interval borders. We consider: 

DS = (v^) -{b-v) . (35) 

11 



We may pass to time-independent drift fields and set 6 = -^, j = fp, / = — VV^ plus 
D = ksT /ui'^. Then: 

S = {S)int + (5)exi (36) 

where 

kBT{S),^t = m^{v^)>0 (37) 

stands for the entropy production rate, while 

kBT{S)ext = - / f ■jdx = -m7 {b ■ v) (38) 

(as long as negative which is not a must) may be interpreted as the heat dissipation 
rate:— f f ■ j dx. 

In view of j = pv = :^[f - fc^TVlnp] = -^V^ i.e. v = -(l/m7)V^ and 
/ = —W, we can introduce 

<iJ = V + kBT\np (39) 

whose mean value stands for the Helmholtz free energy of the random motion 

F={^) = U~TS. (40) 

Here S = ksS and an internal energy is t/ = {V). Since we assume p and pVv to 
vanish at the integration volume boundaries, we get 

F = -(m7) {v^) = -kBT{S)int < . (41) 

Clearly, F decreases as a function of time towards its minimum, or remains constant. 
Let us consider the stationary regime S = associated with an ( a priori assumed 
to exist, p!l]) invariant density p^,. Then, 

b = u = Z^Vlnp* 

and 

- {l/kBT)VV = V In p, ^ p, = i exp[-V/kBT] . (42) 

Hence 

^, = y + A;bT In p, ^ (^,) = -kBT \nZ = F, (43) 

with Z = j exp{~V/kBT)dx. F^ stands for a minimum of the time-dependent Helmholtz 
free energy F. Because of 

Z = exp{-FjkBT) (44) 

we have 

p, = exp[(F, - V)/kBT] (45) 

12 



Therefore, the conditional Kullback-Leibler entropy, of the density p relative to an 
equilibrium density p* acquires the form 

kBTHc = -ksT [ p\n(-^)dx = F, - F . (46) 

J P* 

In view of the concavity property of the function f{w) = —wlnw, Tic takes only 
negative values, with a maximum at 0. We have F^ < F and ksTHc = —F > 0. Tic 
is bound to grow monotonically towards 0, while F drops down to its minimum F* 
which is reached for p^,. The Helmholtz free energy minimum, in the present context 
(and in contrast to the previously described case of discrete probability measures), 
remains divorced from any extremal property of the Gibbs-Shannon entropy. Only the 
Kullback-Leibler entropy shows up an expected asymptotic behavior. See e.g. also 
[HfTH]. 



6 Outlook 

Standard (thermodynamical) notions of entropy are basically introduced under equi- 
librium conditions and are not considered in the time domain. Our discussion was 
tailored specifically to non-equilibrium systems and processes. Any conceivable idea 
of approaching the state of equilibrium, or passing from one such state to another 
(steady) state, always involves the time dependence and the related, often rapid, non- 
equilibrium djTiamical process. 

The major tool invoked in connection with both equilibrium and non-equilibrium 
phenomena is that of Gibbs-Shannon entropy whose definition directly involves time- 
dependent probability distributions. However, let us recall that except for the ther- 
modynamical Clausius case, the very notion of entropy is non-universal and purpose- 
dependent, p2]- Our entropy choice has served a concrete purpose: encompassing a 
temporal behavior of specific probability distributions associated with diffusion-type 
processes. 

The sole entropy methods are neither exclusive nor sufficient to give full account 
of the asymptotic properties of diffusion processes. Additional inputs pertaining the 
regularity properties of solutions of Fokker-Planck equations are necessary to guarantee 
an existence of a stationary solution and to demonstrate that any other solution of the 
pertinent equation must finally decay to the stationary one in the large time asymptotic. 

For standard diffusion-type processes, we have discussed, the standard min/max 
entropy principles do not literally work, [17J. It is the Helmholtz free energy which 
shares proper extremal behavior. On the other hand it is the conditional Kullback- 
Leibler entropy which (together with its time rate) stays in close affinity with the 
Helmholtz free energy of the diffusion process and with the involved entropy production. 

13 



The advantage of our methodology is an exphcit insight into the temporal behavior 
of various thermodynamics functionals whose definition is normally restricted to equi- 
librium(or near-equilibrium) phenomena. The conceptual meaning of the Helmholtz 
free energy, or Gibbs-Shannon entropy is consistently elevated to the time-domain, for 
far from equilibrium systems. The auxiliary notions of work and heat transfer rates 
have received a transparent interpretation as well. 
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